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MONITORING AND ANALYSIS OF DATA IN CYBERSPACE 



^1 




ORIGIN OF INVENTION 



The invention described herein was made 'in the 



performance of work under a NASA contract, and is subject to 
the provisions of Public Law 96-517 (35 USC 202) in which 
the Contractor has elected to retain title. 

FIELD OF THE INVENTION 
This invention relates to a data monitoring and 
analysis system. More specifically, the present invention 
defines a user interface providing a three dimensional 
representation of complex data, data relationships and 
system status. 



operator to monitor status of the system by assimilating a 
large amount of real-time status data received from various 
system components. Complex systems of this type include 
spacecraft operations, industrial plants, financial trading 
activities, and hospital monitoring of medical data from 
patients. All of these systems require important decisions 
to be made, sometimes quickly, in response to a myriad of 



BACKGROUND AND SUMMARY OF THE INVENTION 



The operation of many complex systems requires an 
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data. Once the status of the system or the nature of a 
problem is understood, the system operator takes appropriate 
actions to implement decisions or to remedy the problem. In 
extreme cases, these actions may save the system from 
5 serious undesirable consequences. It is very important that 
this complex data be presented clearly and concisely so that 
the operator can obtain any information needed for 
diagnostic purposes as easily and quickly as possible. 

Existing data display systems often use numerical or 

10 two-dimensional displays which require the operator to 

interpret and analyze the data and its relationship to other 
data. Some of the features of these kinds of displays are 
illustrated in Figures la-lc, which graphically illustrate 
how the composition of total information conveyed to a user 

15 varies over time for various interface types. The total 

information conveyed in these graphs is equal to the amount 
of "navigation data" plus the amount of "destination data" . 
Navigation data is defined as that class of information that 
orients a user in time, space, location and direction; 

20 informs the user of progress in relative and/or absolute 

terms; and that contains addresses, instruction to proceed, 
and/or warnings not to proceed. Destination data is defined 
as that class of information that satisfies the user's 
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desire for information, for example, by answering a 
question. 

Typically, the goal of a user interface is to 
maximize the volume of available destination data while 
5 minimizing the delay in making the destination data 

available. As shown in Figure la, an operator of a command- 
driven interface has very little destination data available 
initially and spends roughly half of the time traversing 
alphanumeric menus ("navigation data") to get to the desired 

10 data display ("destination data"). 

An operator of a menu-driven interface (Figure lb) 
typically can access a larger volume of destination data 
somewhat faster after first navigating through one or more 
menu items. However, the operator can become frustrated 

15 upon encountering several layers of sub-menus before being 
able to access the bulk of the destination data. This 
results in a drop-off (at time D in Figure lb) in the amount 
of available destination data. 

In a graphical user interface ("GUI"), the operator 

20 traverses through various windows and screens to get at the 
destination data in a step-wise fashion as illustrated in 
Figure Ic . Certain destination data may not be accessible 
until the operator has stepped through a number of windows, 



PATENT 

ATTORNEY DOCKET NO: 06816/022001 

thus diminishing the operator's efficiency. These graphs 
were presented by Michael Benedikt in the article 
"Cyberspace: Some Proposals" which appears in the book 
Cyberspace : First Steps . 
5 These shortcomings of command, menu and GUI driven 

interfaces are magnified in data intensive operational 
environments of the kind described above. The inventors 
recognized a need in these environments for an improved user 
interface that makes more information available in less 

10 time. Destination data is one example of such data. These 
problems are well illustrated in the example of spacecraft 
operation control. 

During a typical spacecraft mission, data parameters 
from many different domains are monitored and/or analyzed in 

15 real time to ensure that the spacecraft and its instruments 
are working properly. A mission control site receives this 
data from the spacecraft by telemetry. In this setting high 
volumes of data are received at very frequent intervals. 
Monitoring this data poses a daunting and complex challenge. 

20 This is especially true when multiple spacecraft missions 
are being monitored simultaneously by a single system. 

Early spacecraft data monitoring systems employed a 
panel of discrete, hardwired lights, each light 
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corresponding to an individual data parameter. A team of 
human operators (e.g., flight controllers or analysts) 
monitoring the spacecraft mission, would continually observe 
the various lights to ensure that the spacecraft and its 
5 instruments were operating properly. A trained operator 
could readily determine whether error or alarm conditions 
were present simply by scanning the panel for the 
appropriate color light. For example, a green light might 
mean that a particular data parameter was within normal 

10 ranges, while a red light might indicate that acceptable 
levels for that parameter had been exceeded. The lights 
provided a dramatic and immediately comprehensible 
indication of system status. This early light-panel 
monitoring system hence had the advantage of providing the 

15 operator with an intuitive sense of system health. 

As spacecraft and their associated instruments 
became more sophisticated, light -panel monitoring systems 
became inadequate to handle the volume and complexity of the 
telemetry data that was being collected. Light -panel 

20 systems also were difficult and labor-intensive to adapt for 
new types of telemetry data that changed with each mission. 

The next generation of data monitoring systems 
attempted to solve this problem. A computer system (for 
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example, a work station) received telemetry data and 
displayed it in textual (i.e., alphanumeric) form on the 
display monitor. Figure 2 shows a sample output of a 
typical text -based data monitoring system. The necessarily 
5 high volume of telemetry data causes the display screen to 
become cluttered and difficult to read. An operator relying 
on the text -based monitoring system in Figure 2 is required 
to study and understand large amounts of alphanumeric 
ri information to accurately monitor the health of spacecraft, 

\i 10 That task requires extensive training and considerable 

i j1 

1= diligence on the operator's part. Even with sufficient 

training, using a text-based system to monitor data is 
fatiguing for the operator and frequently results in missed 
alarm conditions or slow responses to alarm conditions. 

15 In short, text -based data monitoring systems do not 

^ generate easily recognizable indicators of changes in system 

status. Rather, a change in system status (even a 
potentially catastrophic one) is signalled merely by 
changing one or more alphanumeric characters to other 

2 0 Iphanumeric characters on a display screen. The screen 
typically contains several hundred such alphanumeric 
characters, and even the most vigilant human operator can 
occasionally overlook an important alarm condition. Thus, 
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there is a need for a system that can present a large amount 
of information to a user in a way that is readily 
understandable and in a way that conveys important 
conditions in a very noticeable way. 
5 The data monitoring system of the present invention 

uses a three dimensional simulated space representation, 
(sometimes called a "cyberspace representation,"), to 
interface with, and to communicate complex, real-time 
information to an operator. A cyberspace representation 

10 utilizes various elements (e.g., time, space, sound, travel, 
and user presence in the computer environment) to convey 
information to the operator. The root word "cyber" comes 
from the Greek word "kybernan" which means "to steer or 
control." Literally, cyberspace means to steer or control 

15 space. 

The cyberspace data monitoring system of the present 
invention may be implemented in conjunction with artificial 
reality or virtual reality systems. The cyberspace 
interface of the invention provides a graphically-oriented 
20 user interface where the operator is figuratively positioned 
within the user interface environment. The invention uses a 
visual representation to communicate information to the 
user. The interface appears to the user as a perspective 
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view of a three-dimensional space from a particular vantage 
point . The user may move around within the three 
dimensional space, and thus change his view through 
manipulation of a cursor input device. As the user 
5 navigates through the three-dimensional space, the display 
changes to reflect the operator's new position within the 
space. This movement may reveal objects that were not 
previously visible from the operator's prior vantage point. 

The use of this cyberspace representation allows the 

10 present data monitoring system to realize a "textless" or 
"minimal text" display that allows a human operator to 
assimilate and comprehend large quantities of data at a rate 
and to a degree much greater than previously possible. This 
is because the user interface of the invention is in a form 

15 similar to that used to comprehend real-world events. 

Rather than representing information in the form of multiple 
rows and columns of alphanumeric characters, data is 
depicted in the form of graphic symbols or objects which are 
positioned within a three-dimensional virtual universe. The 

20 respective dimensions of the virtual universe correspond to 
categories of information from various domains. For 
example, in one embodiment of the invention, these 
categories may include: the various spacecraft missions 
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being monitored, various user-defined categories of data 
parameters that are relevant to the application, attitude 
control system, propulsion, Alarm Log and User-Defined. 
The Alarm Log category records all anomalous data values 
5 until they are acknowledged by the user. The User-Defined 
category allows copies of data values from any other 
category to be grouped together. A third dimension may 
correspond to certain user-defined alarm limits of state 
value for each of the data parameter categories. 

10 The shape, motion, position, color and behavior of 

the graphic symbols are configured to convey an extensive 
amount of information about the current values of the data 
parameters to the human operator in an easily understandable 
and instantly recognizable manner. 

15 Figure Id shows the Total Information to User graph 

for a cyberspace -driven interface. A cyberspace -driven 
interface does not require an operator to enter alphanumeric 
commands or to traverse menu items or windows before 
destination data is available. Rather, as illustrated in 

2 0 Figure Id by the three exemplary curves of cyberspace -driven 
interfaces, the operator has essentially instantaneous 
access to a considerable amount of destination data. This 
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results in a greater amount of usable information being 
conveyed to the operator in less time. 

Another advantage provided by the cyberspace -driven 
system of the invention is that its visual abstraction of 
data allows an operator to assess the qualitative status of 
the various spacecraft with not much more than a mere glance 
at the display screen. For example, in one embodiment of 
the cyberspace system of the invention, information about 
alarm severity is conveyed by the height of a pole extending 
from the graphical objects. In a text-based system, in 
contrast, severity information may be obtained only by 
manually referring to the alphanumeric alarm limits of the 
parameters of interest. 

The cyberspace system also permits more data to be 
displayed on the screen at any one time. Only a part of one 
screen is used in the cyberspace system (in contrast to the 
multiple screens that are used for text-based systems) , 
hence all of the telemetry data for an entire system can be 
displayed unambiguously in a fraction of the screen real- 
estate that has been previously required. 

Yet another advantage of the cyberspace data 
monitoring system is that it allows multiple systems to be 
monitored simultaneously, and further allows related data 

- 10 - 
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parameters to be grouped (and re -grouped) by the operator 
logically in space. Thus, the cyberspace system is easily 
tailored to the specific needs of a particular system or to 
the preferences of a particular operator. 
5 Standard computer graphics manipulation techniques 

are used to facilitate navigation through this virtual 
universe of data to alter the relative vantage point within 
the universe. Hence the operator may move between different 
monitored systems (such as different spacecraft missions) to 

10 inspect a particular data parameter or group of parameters 

in more detail than might be possible at a different vantage 
point within the universe. 

The cyberspace data representation provided by the 
invention eliminates the need for an operator to read large 

15 volumes of textual material in order to determine system 

status. Rather, merely by scanning the attributes of the 
graphic symbols displayed on the screen, an operator gleans 
an intuitive understanding of the system status that extends 
far beyond the knowledge that could be imparted by a visual 

20 inspection of either a light-panel display or a text-based 
display. 

Other- advantages of the cyberspace data monitoring 
system include the following: an operator is able to 
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recognize impending alarm conditions earlier and with 
greater reliability due to the intuitive nature of the 
cyberspace interface; operators may be trained faster and 
with greater efficiency; a plurality of sub-systems can be 
5 operated as a single system instead of as collections of 
sub-systems because the status of the entire system can be 
understood from one display; fewer operators may be required 
by enabling a single operator to effectively monitor larger 
components of a system; operators can become aware of 
10 problem situations before actual alarms occur and without 
needing to manually assess collected trend data; and 
operations differences between non- identical systems can be 
minimized. 



BRIEF DESCRIPTION OF THE DRAWINGS 
15 These and other features and advantages of the 

present invention will become evident from the detailed 
description set forth hereafter with reference to the 
accompanying drawings . 

Figures la- Id are graphs of Total Information to 
20 User versus Time for various interface types. 

Figure 2 is a screen display from a text-based data 
monitoring system; 

- 12 - 
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Figure 3 is a block diagram of a preferred 
embodiment of the cyberspace data monitoring system of the 
present invention; 

Figure 4 is a screen display from the cyberspace 
5 data monitoring system of Figure 3; 

Figure 5a is a screen display of the overview window 
in the cyberspace data monitoring system of the present 
invention; 

j Figure 5b is a screen display of the main window in 

J 10 the cyberspace data monitoring system of the present 
= invention; 

Figure 6 is a screen display from the cyberspace 

i 

' data monitoring system of the present invention; 

: Figure 7 is a screen display from the cyberspace 

{ 15 data monitoring system of the present invention; 
^ Figure 8 is a pop-up display from the cyberspace 

data monitoring system of the present invention; 

Figure 9 is a screen display of the alarm 
suppression facility of the present invention; 
20 - Figure 10 is a screen display of a suppressed object 

in accordance with the present invention; and 

Figure 11 is a flowchart of the method of one 
embodiment of the invention. 

- 13 - 
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DETAILED DESCRIPTION 
Figure 3 shows a preferred embodiment of the 
cyberspace data monitoring system (CDMS) 10 of the present 
invention. The CDMS 10 has special features which 
facilitate monitoring and control of one or more complex 
monitored systems 14, 16 and 18 by an operator. The 
operator may be required to quickly assess the real-time 
status of the monitored systems 14-18 and to make 
appropriate decisions regarding necessary responses to the 
system status. 

Data from the monitored systems 14-18 is processed 
by a conventional host computer 20 which includes a CPU 22, 
memory 24 and 10 unit 26. Host 20 may preferably comprise a 
UNIX or PC workstation. The host 20 also includes a 
cyberspace user interface 28 which contains the necessary 
computer programs and/or operating systems operating to 
implement the techniques of the invention. A transmitting 
unit such as a modem 29 is optionally employed for notifying 
an operator of a situation by means such as paging, in 
addition to a visual display. 

The cyberspace user interface 2 8 receives 
monitored system data under control of the CPU 22 and 
organizes and configures this information for display in 

- 14 - 




PATENT 

ATTORNEY DOCKET NO: 06816/022001 



accordance with the teachings of the present invention. In 
particular, the cyberspace user interface 28 controls a user 
interface module 3 0 containing a display unit such as a CRT 
32 and one or more input devices (keyboard, mouse, etc.) 34. 
The input devices 34 are controlled by the user in response 
to the information on the display unit 32. This response is 
then transferred to the cyberspace user interface 28 for 
processing and modifying the display on the display unit 32. 
Additional information from the input devices may also 
include commands sent to the host 20 for control of the 
monitored systems 14-18. 

The CDMS 10 was developed by the inventors using the 
C programming language and the Open GL graphics library. It 
was implemented on Silicon Graphics hardware, and may run on 
other UNIX and PC work stations. It should be understood, 
however, that any graphical user interface operating system 
could form the backbone for this system. 

Figure 4 is a screen display from the CDMS 10 in 
accordance with a preferred embodiment of the present 
invention. The invention will be described in detail using 
an embodiment that is adapted to the task of monitoring a 
plurality of spacecraft. It will be appreciated that this 



- 15 - 
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application is one of many in which the teachings of the 
present invention may be employed. 

Data regarding a plurality of spacecraft missions 
are depicted in Figure 4 in the form of graphic symbols or 
objects which are positioned within a three dimensional 
virtual universe. The respective dimensions of the virtual 
universe correspond to a variety of aspects of the 
spacecraft missions 36. Along a first dimension is an 
identification of the source of displayed data parameters, 
such as various spacecraft missions being monitored, labeled 
0712, 4487, 4524, etc. Along a second dimension appears 
various user-defined categories of data parameters that are 
relevant of the application 38, including Communications 
(COMM) 38, Power 40, Attitude Control System (ACS) 42, 
Propulsion (PROP) 44, and SCT 46. Two additional categories 
that can be used in most applications are Alarm Log 48 and 
User-Defined 50. The alarm log records all anomalous data 
parameter values until they are acknowledged by the user. 
The user-defined grouping allows copies of data values from 
any combination of data parameter categories to be grouped 
together. A third (vertical) dimension corresponds to 
certain user-defined alarm limits and state values for each 
of the data parameter categories. 
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The configuration and position of the graphic 
symbols in Figure 4 convey a extensive amount of information 
about the current state of the data parameters to the human 
operator 12 in an easily understandable and instantly 
5 recognizable manner, as described below. 

Figures 5a and 5b show the start-up screen of the 
CDMS 10 which displays two windows: an overview window 56 
(Figure 5a) and a main window 54 (Figure 5b) . The overview 
window provides the operator with a global view of the 

10 entire cyberspace universe. All data parameter information 
for the missions being monitored is visible in the overview 
window at all times, thus keeping the operator apprised of 
all alarm conditions occurring anywhere within the 
cyberspace universe. The overview window also allows the 

15 operator to know the location of the current viewpoint 

within the main window, as denoted by a green dot 52 that 
tells the operator, "you are here." 

The main window 54 shown in Figure 5b constitutes 
the primary source of information for the operator. Unlike 

20 the overview window that always displays the entire 
universe, the main window will typically (but not 
necessarily) display only a subset of the entire universe at 
any one time. An operator navigates through three- 

- 17 - 
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dimensional cyberspace using a cursor control device 
included in input devices 34 (such as a mouse or a 
trackball) to view the data from any desired viewpoint or 
perspective. This shifting of viewpoint is accomplished 
using software techniques commonly employed in computer 
graphics products such as computer aided design (CAD) 
systems. The particular viewpoint shown in Figure 5b is a 
perspective view of the cyberspace universe from a position 
slight above and pulled back from the universe. A top 
orthogonal view of a representative cyberspace universe is 
illustrated in window 58 in Figure 6. 

The main window 54 includes a white grid 60 divided 
into rows and columns to form adjacent grid squares. The 
rows correspond to different spacecraft missions (e.g., 
MAGELLAN, VOYAGER 1, VOYAGER 1) and the columns correspond 
to user-defines data parameter categories within each 
mission (ALARMS, SSO FAULT PROTECTION DATA, etc.) . These 
categories are an application specific way of categorizing 
the data. The white grid 60 corresponds to the baseline or 
nominal values of the data parameters . A green upper grid 
62 is suspended above and in registration with the white 
grid to represent the flight approved error ceiling -- i.e., 
the user-defined critical values for the various data 

- 18 - 
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parameters beyond which serious consequences, such as 
permanent damage to the monitored system, occurs. 

The geometric objects within the white grids of the 
display represent individual spacecraft telemetry data 
parameters . Each of the data parameters has a number of 
attributes which are represented. For example, each of 
three possible shapes — rectangle, diamond, and triangle — 
indicate a different type of data parameter; analog, 
digital, and counter, respectively. Depending on the 
application, these and other graphical shapes may be used to 
indicate other attributes and qualities about the data 
parameter represented. The position of an object on the 
grid can also indicate its functional classification via 
adjacency to related parameters, if the user elects to group 
the objects within the grid square in some meaningful way. 
For example, objects may be grouped by type of subsystem 
(electrical, hydraulic, mechanical) or by physical location 
within the system. All data parameters are normalized to 
allow a diverse mix of data types and units. 

While viewing the data objects, the operator may 
dynamically and interactively specify logical groups of 
parameters for monitoring. Each user-defined grouping (38- 
46) of parameters defines a parameter category which is 
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delimited by a separate column in the grid. The operator 
may define categories through the use of configuration 
tables or pop-up menus. Permanent categories such as "Alarm 
Log" or "User-Defined" may also be implemented. 

The User-Defined category 50 is a temporary 
collection of data parameters that the operator is likely to 
periodically change at various stages during the mission 
under consideration. One example of such a transient usage 
of a user-defined grouping would occur when a particular 
subsystem in one spacecraft (mission) fails or shows some 
anomaly. The operator may then want to group all data 
parameters relevant to the problem (even from multiple 
categories) and closely monitor those specific parameters 
for each mission in the User-Defined column. In general, 
anytime a given situation suggests that certain parameters 
demand close attention, the operator can easily place those 
parameters temporarily into the User-Defined column. Hence 
parameters which have not reached an alarm or critical state 
can be closely monitored during a critical period. The 
Alarm Log category is a redundant collection of all alarms 
presently occurring in the mission, without regard to 
particular data parameter categories. 



- 20 - 




PATENT 

ATTORNEY DOCKET NO: 06816/022001 



In this interactive environment of three dimensional 
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graphical objects in cyberspace, the operator can quickly 
comprehend the full range of information contained in the 
telemetry data by looking at combinations of shape, color, 
position, and behavior of geometric objects. 

Anomalous data parameter states --i.e. alarm 
conditions-- are immediately visible to the viewer through 
the color and behavior of the objects. Two types of alarms 
are employed in this embodiment of the cyberspace system: 
limit alarms and trend alarms. A limit alarm occurs when 
the value of a parameter is outside a predefined pair of 
high and low limits. For example, a limit alarm is 
activated when one of the following situations occur: (i) an 
analog value falls outside of its predefined nominal range; 
(ii) the bit state of a digital parameter does not 
correspond to a predefined state; or (iii) the counter value 
of an ASCII data parameter does not correspond to a 
predefined value. 



(preferably yellow) and critical (preferably red) . Figures 
4, 5a-b show these objects. Objects are displayed in the 
default color, white, to indicate that the parameter it 
represents is in its normal state. Objects are displayed 



Two levels of limit alarms are used: warning 
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yellow or red to indicate whether the parameter is in a 
warning limit alarm state (yellow) , or a critical limit 
alarm state (red) . Figures 4 and 5a- 5b also show that 
during warning or critical states, the graphical object 
5 rises or elevates, to a height above the nominal grid. This 
height is proportional to how far the parameter has exceeded 
the limit. This provides a graphic indication of the 
severity of the alarm. The heights to which the objects 
rise are normalized so that the objects always remain 

10 positioned between the lower and upper grids, inclusive. At 
the same time that the object rises, a bar or pole extends 
from to object to the lower grid. This pole serves to 
assist the user in estimating the distance that the object 
has risen. Since the length of the pole is proportional to 

15 the absolute value of the data parameter. 



the visibility of the alarm state. As clearly illustrated 
in Figure 5a, this is very useful when viewing objects from 
an overhead perspective where the length of the pole below 
2 0 the object may be obscured. In such cases only the color is 
available to indicate the presence and severity of the 
alarm. A user can recognize that an alarm is active by an 
object's color, and can easily change perspective to a side 



The change in color of an object serves to increase 
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view to clearly see the height of the object and associated 
pole . 

As shown in Figures 4 and 5a- 5b, objects in the 
"Alarms" category (equivalent ly, "Alarm Log") do not rise 
5 above the normal grid when they enter an alarm state. This 
is preferred because if these alarm objects were elevated as 
in other columns, it would produce a display that was too 
dense with elevated objects for a user to easily intuit 
meaning from height . 

10 If the pole for a data object reaches the upper grid 

62 it means that a threshold level of severity of the alarm 
situation has been reached; for example, it may mean that an 
unrecoverable error has occurred. When this happens, the 
color of the corresponding square in the green upper grid 62 

15 changes, from green to red. The object will remain the same 
color and shape but will now be encircled by a red square 
superimposed over that region in the green upper grid 62. 
This provides the operator with a dramatic indication that a 
system error has occurred which may need immediate 

20 attention. When a yellow object reaches the upper grid it 

will change to red and begin rising from the lower grid at a 
level proportionate to its data parameter value. 
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Rather than depending on the current value of a data 
parameter, a trend alarm is based on the rate of change in a 
parameter value over time that indicates some kind of 
pattern. If the rate of change of a parameter exceeds some 
5 predefined limit over a predefined period of time, then the 
CDMS 10 triggers a trend alarm for that parameter. A trend 
alarm may be triggered when a data value steadily increases 
or decreases during a specified time range. Or, the trend 
alarm may be triggered when the rate of change for a data 

10 parameter has exceeded a predefined limit. For example, if 
a fire starts in a spacecraft a temperature sensor may show 
an unusually rapid increase in temperature. This rapid rate 
of increase will likely trigger a trend alarm well before 
the temperature value reaches an alarm level. This gives 

15 extra time to respond to the fire and mitigate the damage. 

The CDMS 10 currently uses two different techniques 
for detecting trend alarms. The first trending technique 
tracks the continual increase or decrease of the data 
parameter value over a user specified time interval in which 

20 a warning or critical alarm will occur. For example, a user 
may specify (in the mission database) that a critical alarm 
should occur if spacecraft cabin pressure decreases for more 
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than 8 seconds, because this may indicate a cabin leak as 
opposed to normal pressure fluctuations. 

The second trend alarm involves tracking the rate of 
change over time: if the rate change of a parameter exceeds 
5 some predefined limit over a specified period of time, then 
the parameter triggers a trend alarm. For example, a rapid 
non- transitory drop in a battery voltage may suggest a short 
circuit rather than normal electrical loads. Depending on 
the system's objectives, a variety of other trending 

10 techniques may also be used. The CMDS 10 performs trend 
analysis in real-time. 

Both of these types of trend alarms can assume one 
of two levels of severity, each of which is represented 
through a unique data abstraction. If the CDMS 10 

15 determines that the trend alarm is less severe (a "warning 
level" trend alarm) , the graphical object associated with 
that parameter will appear to spin on the screen display. 
If the alarm is more severe (a "critical level" trend 
alarm) , the object will repeatedly flash between different 

20 levels of brightness. 

In any given spacecraft mission, a particular data 
parameter may experience a limit alarm and a trend alarm 
concurrently. As noted above, a limit alarm and a trend 
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alarm may each occur in one of two severities -- a limit 
alarm may be either warning level (yellow) or critical level 
(red) and a trend alarm may be either warning level 
(spinning) or critical level (flashing) . A preferred 
embodiment of the CDMS 10 employs the unique combinations of 
graphical techniques identified in Table I to represent each 
of nine potential combinations of alarm states. 

TABLE I 





No 


Warning 


Critical 




Trend 


Trend 


Trend 




Alarm 


Alarm 


Alarm 


No 


white 


brown 


brown 


Limit 


not spinning 


spinning 


not spinning 


Alarm 


not flashing 


not flashing 


flashing 


Warning 


yellow 


yellow 


yellow 


Limit 


not spinning 


spinning 


not spinning 


Alarm 


not flashing 


not flashing 


flashing 


Critical 


red 


red 


red 


Limit 


not spinning 


spinning 


not spinning 


Alarm 


flashing 


not flashing 


flashing 



If a trend alarm occurs while the object already is 
in a limit alarm state, the color of the object remains the 
same as for the existing limit alarm (i.e., either red or 
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yellow) . If , on the other hand, the object is not in a 
limit alarm state, it turns from its nominal white color to 
brown when only a trend alarm is triggered. Hence a brown 
object immediately tells the user that a trend alarm, but no 
limit alarm, is active. This use of color, spinning, and 
flashing makes possible the unambiguous display of all 
possible combinations of limit and trend alarms in a single 
obj ect . 

Operators familiar with the CMDS 10 will be able to 
immediately assess which of the 9 different conditions exist 
for each parameter. As demonstrated by Figure 4, several 
hundred data parameters are easily visible at the same time. 
This means that the operator can quickly assess thousands of 
possible conditions at a glance. Also, more serious 
conditions receive more prominent displays. 

Furthermore, as indicated by Figure 4, more distant 
grid lines converge into a distant virtual background space. 
This compresses the more distant data representations into a 
smaller portion of the screen space. Regardless of the user 
viewpoint, when moving through this virtual space, the data 
of greater interest is displayed closer and larger than data 
of lesser interest which is displayed smaller and farther 
away. This distance can continue indefinitely without any 
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theoretical limit. Hence, at extreme virtual distances 
objects in the background will be so small that the user may 
not be able to ascertain much more than the existence of an 
alarm status. The user will then need to move his virtual 
5 viewpoint much closer in order to determine exactly which 
parameter is in an alarm state. 

By shrinking less relevant data representations, the 
user is able to effect a continuously variable choice of how 
^1 much detail of the various regions of data are displayed at 

Si 10 a given moment. At the same time, the user can maintain an 
overall view of the entire data field. Since there is no 
theoretical limit on how far the grids and objects may 
extend into the virtual distance, the present invention 
makes it possible to monitor a seemingly limitless amount of 
15 data at a given time. For example, upper and lower grids 60 
=^ and 62 can extend indefinitely in all directions into 

virtual space. Multiple sets, or arrays, of such grids can 
be displayed on a single screen as viewed from a very large 
virtual distance. To effectively monitor such extremely 
20 fine-grained displays may require increasing the visibility 
of the representation of alarm states because such distant 
objects will be extremely small. In contrast, prior 

display and monitoring systems do not provide this intuitive 
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and qualitative display of such potentially large amounts of 
data and would require much more screen space and multiple 
screens to communicate this much information. 

An operator may learn additional information about 
parameters that are in alarm (i.e., beyond the information 
that is already graphically represented) by clicking on the 
data object of interest. This action causes a pop-up window 
64 to appear below the main window as shown in Figure 7. 
The pop -up window contains text information concerning the 
chosen parameter and all other parameters contained within 
the same parameter category. As shown in the more detailed 
view of the pop up window 64 in ^ Figure 8, each row 
corresponds to a data parameter. Detailed information 
includes the data parameter mnemonic, data type, 
description, time of receipt, floating point value, data 
value as received, and alarm limits. Of this information, 
only a limited amount of information (such as data type, 
data value and alarm limits) can be obtained from the 3D 
display 54; and this information is represented 
qualitatively rather than quantitatively. The detailed 
numeric information is only needed in unusual circumstances 
and therefore is not included in the 3D display 54 . 
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Some of the information in the pop up window 64 is 
similar to what has been traditionally available in text- 
based displays: parameter name, time of last update, 
engineering unit value or data number, and type of alarm. 
5 Other information available in the pop up window 64 relates 
to new types of alarms associated with various types of 
trend analysis. Still further information such as the 
warning and critical limits of the trend algorithms may be 
available in the pop-up window 64. Text-bound displays 
10 would require an operator to view a different screen for 
each of possible several hundred objects to glean the 
equivalent information that is available at a glance using 
the CDMS 10. 

The default amount of information that is displayed 
15 in both the graphic windows (i.e., the main 54 and overview 
56 windows) and in the pop up text windows 64 is user- 
selectable based on operator expertise and preferences. The 
default configuration can be dynamically adjusted by the 
operator during monitoring operations to reflect changing 
20 needs. As shown in Figure 6, for example, the operator may 
configure the system to display only two of the three 
potentially available missions or to add a desired number of 
additional missions. Configuring the display involves 
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deleting or adding mission database to or from the data 
directory. 

The CDMS 10 also may be used for off-line for non- 
real-time analysis. Historical data files can be replayed 
5 and trend data that has been saved to a database can be 
displayed. Other features (available in real-time or non 
real-time mode) allow the display of conventional plots or 
trend data, suppression of nominal parameter values, 
suppression of alarm notification for user-specified time 

10 intervals, and movement of data objects to any grid-square 
location chosen by the operator. Suppression of alarms can 
be done by clicking on the desired data parameter and 
entering a time unit the suppression is deactivated (as 
shown in Figure 9) . The suppression of the object is 

15 conveyed visually (as shown in Figure 10) by turning the 

object blue. The system automatically detects when the time 
expires and turns the object back to its original state. 
This feature can be used for data parameters which are known 
by the operator or analyst to be anomalous. Organization of 

2 0 data parameters is achieved by selecting the object to move 
and clicking on the desired area in the grid square where 
the object will be placed. 
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The highly visual look and feel of the CDMS 10 may 
be complemented with the use of speech output to communicate 
key alarm and parameter identification information to the 
operator without requiring the reading of text or the 
viewing of graphic objects. A high-quality audio system is 
preferably used to digitize and edit key words and phrases. 
When an operator clicks on an object representing a data 
parameter or when an alarm occurs, the CDMS 10 joins 
appropriate words and phrases to deliver a voige alarm 
(i.e., a spoken message) to the operator indicating the 
identity of the parameter, the type of alarm (trend or 
limit) and its severity. When whole words are digitized the 
resulting speech has a smooth and natural sound. If a more 
flexible vocabulary is required other speech synthesis 
methods such as phoneme-based speech synthesis may be used. 
When a particular data parameter (e.g., E-1234) is in a 
warning state, the system may say "Parameter E-1234 is in 
warning." To alert the operator of multiple alarms, the 
system may say "Four critical alarms have been detected, " 
thereby informing the operator of the number and severity of 
alarms occurring. 

The CDMS 10 also may be enhanced by providing a 
speech recognition capability. Efficient pattern matching 
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algorithms can be used to accurately identify commands 
spoken by the operator. Standard techniques may be used in 
which the operator's speech is filtered into frequency bands 
that are further subdivided and compared with previously 
5 digitized samples used to "train" the software. 

Other features which may be used in other 
embodiments of the CDMS 10 include the following: 

(i) The system may be configured to automatically 
dial the operator's telephone pager number by means of a 

10 modem 29 whenever an alarm condition occurs, thus informing 
the operator that the monitoring system requires attention. 
As a result, the operator need not be physically present at 
the monitoring system site at all times. Any standard 
telephone paging device will suffice for this purpose. The 

15 paging feature may be enabled by entering the pager number 
into a text-field of the parameter table. 

(ii) The system may employ plotting capabilities 
which may be used to plot data parameter objects against 
real time, historical averages; and summary data, such as, 

20 minimum, mean, maximum, standard deviation, etc. The 

operator may enable plotting through a point -and-click menu. 
Any conventional plotting hardware may be used. 
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(iii) Output data from the system may be printed in 
either ASCII or Postscript format. 

(iv) The system may provide an extensive and user- 
friendly on-line help system that guides the operator in a 
clear and succinct step by step format. This may be 
implemented using a hypertext browser such as the one 
employed in NETSCAPE. 

The implementation of cyberspace representation is 
highly generic --that is, it is applicable to a wide variety 
of applications in which varying data is monitored. This 
enables it to be used for a wide variety of simultaneous 
data streams and/or missions. Other applications include 
sensor data from industrial plants, time varying financial 
data (i.e., stock or securities prices), monitored medical 
data i.e., blood pressure, heart rate, etc., from large 
numbers of intensive care patients or other hospital 
patients) . In addition, the cyberspace architecture enables 
the various modules that comprise the system to be 
distributed over a network, if desired. 

Figure 11 shows a flowchart of a preferred 
embodiment of the method of the cyberspace data monitoring 
system of the present invention. The process of the 
invention begins with the generating of a display of a 
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three-dimensional cyberspace universe including upper and 
lower grids as shown in block 66. Next, in block 68 objects 
representing data parameters are arranged in predetermined 
locations based on data categories in the lower grid. The 
5 system configures each object in a predefined geometric 
shape based on data parameter type. As shown in block 72, 
the system determines all current object alarm states. As 
shown in block 74, the system displays all objects with no 
alarm states in the color white. The system displays 

10 objects with only trend alarms active in the color brown, as 
shown in block 76. All objects with a warning trend alarm 
are displayed spinning, as shown in block 78. The system 
then displays all objects with critical alarm trends 
flashing, as shown in block 80. All objects with warning 

15 limit alarms are displayed yellow and elevated to a 

proportional amount by a yellow pole as in block 82. All 
objects with critical alarm limits are colored red and 
displayed elevated to a proportional level by a red pole as 
in block 84 . 

20 As shown in block 86, when a critical alarm reaches 

a highly severe state, as indicated by reaching the level of 
the upper grid, the upper grid square encircling the object 
is highlighted in red. In decision block 90 the system 
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determines whether a user input is present. If so, the 
system determines, if this user input is a request to modify 
the viewpoint, as shown in decision block 94. If not, 
decision block 90 directs the process to decision block 92. 
5 If the user input is a request to modify the viewpoint, the 
viewpoint is altered in a manner which expands approaching 
regions and shrinks receding regions of the display 
proportionately. If a user input to modify the viewpoint is 
not present, as determined in decision block 94, the process 

10 proceeds to decision block 98. Also, once the process 

completes the step shown in block 96, the process proceeds 
to decision block 98, where it is determined whether an 
input is received to display more information regarding an 
object. If so, the status screen 56 is displayed, as shown 

15 in block 100. If not, the process proceeds to decision 

block 102, which is reached also after decision block 100 is 
performed. 



whether a user input is a request to modify user-defined 
20 categories. If so, the objects requested are rearranged 
into the user-defined category in step 104. The process 
then proceeds to block 92; this is also the destination if 
decision block 102 determined there was no input to modify 



Decision block 102 shows the step of determining 



- 36 - 





PATENT 

ATTORNEY DOCKET NO: 06816/022001 



the user- defined category. Block 92 determines whether a 
new alarm condition exists anywhere in the display and, if 
so, directs the process back to step 72 to update all of the 
alarm conditions. If not, the process proceeds to block 106 
5 where it is determined whether there is a new user input. 

If so, the process proceeds back to block 94 to process this 
user input. Otherwise, the process proceeds to block 106 
where it is determined whether there is a stop command, and 
if not, the process returns to decision block 92. If there 
10 is a stop command the process ends. 



implementing the techniques of the present invention. Some 
of these functions may be implemented in hardware or 
software, or a combination of both. However, preferably, 

15 the invention is implemented in computer programs executing 
on programmable computers such as host computer 20 shown in 
Figure 3. Program code is applied to input data to perform 
the functions described herein and generate output 
information. The output information is applied to one or 

20 more output devices such as the display unit 32, in known 
fashion. 



level procedural or object oriented programming language to 



The invention includes various computer programs 



Each program is preferably implemented in a high 
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communicate with a computer system. However, the programs 
can be implemented in assembly or machine language, if 
desired. In any case, the language may be compiled or 
interpreted language . 

Each such computer program is preferably stored on 
storage media or device (e.g. ROM or magnetic diskette), 
readable by a general or special purpose programmable 
computer, for configuring and operating the computer when 
the storage media or device is read by the computer to 
perform the procedures described herein. The inventive 
system may also be considered to be implemented as a 
computer- readable storage medium, configured with a computer 
program, where the storage medium so configured causes a 
computer to operate in a specific and predefined manner to 
perform the functions described herein. 

Although the invention has been described in detail, 
it should be understood that various changes, substitutions 
and alterations can be made without departing from the 
teachings thereof. For example, two-dimensional planes and 
landmarks may be represented by ways other than grids, such 
as by solid objects. More than two grids may be employed, 
and many different shapes, colors and movements, may be used 
to represent data, data trends and data conditions. 
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All such modifications are intended to be 
encompassed within the following claims. 
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